Speaker recognition has many applications and became increasingly prevalent in numerous voice-based communication including entertainment, marketing and customer services, medical transcription, etc. In conventional speaker identification, speakers are often misidentified due to lack of context information on the speakers and speeches, and due to limited amount of information conveyed with speaker labels.